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Abstract — Low-Density Parity-Check (LDPC) codes are 
usually decoded by running an iterative belief-propagation, 
or message-passing, algorithm over the factor graph of the 
code. The traditional message-passing schedule consists of 
updating all the variable nodes in the graph, using the same 
pre-update information, followed by updating all the check 
nodes of the graph, again, using the same pre-update in- 
formation. Recently several studies show that sequential 
scheduling, in which messages are generated using the lat- 
est available information, significantly improves the conver- 
gence speed in terms of number of iterations. Sequential 
scheduling raises the problem of finding the best sequence of 
message updates. This paper presents practical scheduling 
strategies that use the value of the messages in the graph 
to find the next message to be updated. Simulation re- 
sults show that these informed update sequences require 
significantly fewer iterations than standard sequential sched- 
ules. Furthermore, the paper shows that informed schedul- 
ing solves some standard trapping set errors. Therefore, it 
also outperforms traditional scheduling for a large numbers 
of iterations. Complexity and implementability issues are 
also addressed. 

Index Terms — Belief propagation, message-passing sched- 
ule, error-control codes, low-density parity-check codes. 



I. Introduction 

Belief Propagation (BP) provides Maximum-Likelihood 
(ML) decoding over a cycle-free factor-graph representa- 
tion of a code as shown in [1] and [2]. In some cases, BP 
over loopy factor graphs of channel codes has been shown 
to have near ML performance. BP performs well on the bi- 
partite factor graphs composed of variable nodes and check 
nodes that describe LDPC codes. 

However, loopy BP is an iterative algorithm and there- 
fore requires a message-passing schedule. Flooding, or 
simultaneous scheduling, is the most popular scheduling 
strategy. In every iteration flooding simultaneously up- 
dates all the variable nodes (with each update using the 
same set of pre-update data) and then, updates all the 
check nodes (again, with each update using the same pre- 
update information). Recently, several papers have ad- 
dressed the effects of different types of sequential, or non- 
simultaneous, scheduling strategies in BP LDPC decoding. 
The idea was introduced as a sequence of check-node up- 
dates in [3] and [4] and as a sequence of variable-node up- 
dates in [5]. It is also presented in [6] under the name of 
Layered BP (LBP), in [7] as serial schedule, in [8] as shuf- 
fled BP, in [9] as row message passing, column message 
passing and row-column message passing, among others. 
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Simulations and theoretical tools in these works show 
that sequential scheduling converges twice as fast as flood- 
ing when used in LDPC decoding. It has also been shown 
that sequential updating doesn't increase the decoding 
complexity per iteration, thus allowing the convergence 
speed increase at no cost [9]. In [10], where a global frame- 
work for the analysis of LDPC decoders is introduced, the 
complexity is assumed to be independent from the sequen- 
tial scheduling chosen. Furthermore, different types of se- 
quential schedules such as sequential check-node updat- 
ing, sequential variable-node updating and sequential mes- 
sage updating have very similar performance results [9]. 
Given their similarities, the different types of sequential 
updates will be referred in this paper as Standard Sequen- 
tial Scheduling (SSS). In the simulations presented in this 
paper the SSS strategy used for comparison is LBP, a se- 
quence of check-node updates, as presented in [4] and [6]. 

Sequential updating poses the problem of finding the or- 
dering of message updates that results in the best conver- 
gence speed and/or code performance. The current state of 
the messages in the graph can be used to dynamically up- 
date the schedule, producing what we call an Informed Dy- 
namic Schedule (IDS) and presented in [11]. To our knowl- 
edge, the only well defined informed sequential scheduling 
is the Residual Belief Propagation (RBP) algorithm pre- 
sented by Elidan et al. in [12]. They proposed it for general 
sequential message passing, not specifically for BP decod- 
ing. 

RBP is a greedy algorithm that organizes the message 
updates according to how different is the message generated 
in the current iteration from the message generated in the 
previous iteration. The intuition is that the bigger this 
difference, the further from convergence this part of the 
graph is. Therefore, propagating this message first will 
make BP converge at a higher speed. 

Simulations show that RBP LDPC decoding has a higher 
convergence speed than SSS but its error-rate performance 
for a large enough number of iterations is worse. This be- 
havior is commonly found in greedy algorithms, which tend 
to arrive at a solution faster, but arrive at the correct solu- 
tion less often. We propose a less-greedy IDS in which all 
the outgoing messages of a check-node are generated simul- 
taneously. We call this IDS node-wise RBP. It converges 
both faster and more often than SSS. 

Both RBP and node- wise RBP require the knowledge of 
the message to be updated in order to pick which message 
to update. This means that several messages are com- 



puted and not passed. Thus, increasing the complexity of 
the decoding per iteration. We propose using the min-BP 
check-node update algorithm explained in [13] and [14] to 
simplify the ordering metric and significantly decrease the 
complexity for both informed scheduling strategies while 
maintaining the same performance. Also, an analysis of 
the hardware issues that may arise in a parallel implemen- 
tation of these informed sequential scheduling strategies is 
presented. 

This paper is organized as follows. Section II reviews 
LDPC decoding and the flooding and SSS schedules. Sec- 
tion III explains how to implement RBP decoding for 
LDPC codes. This section also introduces and justifies 
node- wise RBP. Section IV addresses some complexity and 
implementability issues. The simulation results of all the 
message-passing schedules arc compared and discussed in 
Section V. Section VI delivers the conclusions. 

II. BP DECODING FOR LDPC CODES 

In general, BP consists of the exchange of messages be- 
tween the nodes of a graph. Each node generates and 
propagates messages to its neighbors based on its current 
incoming messages. The vector of all the messages in the 
graph is denoted by m and mk denotes the fc'th message 
in the vector m. The function that generates irik from m 
is denoted by fk (m). 

The LDPC code graph is a bi-partite graph composed 
by variable nodes Vj for j e {1, . . . , N} that represent 
the codeword bits and M check nodes Cj for « G {1, . . . , M} 
that represent the parity-check equations. The exchanged 
messages correspond to the Log-Likelihood Ratio (LLR) 
of the probabilities of the bits. The sign of the LLR in- 
dicates the most likely value of the bit and the absolute 
value of the LLR gives the reliability of the message. In 
this fashion, the channel information LLR of the variable 
node Vj is Cy- = log , where yj is the received 

signal. Then, for any Cj and Vj that are connected, the two 
message generating functions, mvj^a = fvj^a (m) and 
nici^vj = fci^vj (m), are: 

m„^.^ci = X] fnca^vj+C^^, (1) 

mc,^y^ = 2 X atanh [| tanh {p!^^ , (2) 
\vbeN'{ci)\vj ) 

where M {vj) \ci denotes the neighbors of vj excluding c,, 
and M (ci) \vj denotes the neighbors of c, excluding Vj. 

BP decoding consists of the iterative update of the mes- 
sages until a stopping rule is satisfied. In flooding schedul- 
ing, an iteration consists on the simultaneous update of all 
the messages rriy^c followed by the simultaneous update of 
all the messages nic^y. In SSS, an iteration consists on the 
sequential update of all the messages m,^^c as well as all 
the messages ruc^y in a specific pre-defined order. The al- 
gorithm stops if the decoded bits satisfy all the parity-check 
equations or a maximum number of iterations is reached. 



III. Residual Belief Propagation (RBP) 

RBP, as introduced in [12], is an informed scheduling 
strategy that updates first the message that maximizes an 
ordering metric called the residual. A residual is the norm 
(defined over the message space) of the difference between 
the values of a message before and after an update. For a 
message mfe, the residual is defined as: 

r {mk) ^ Wfk {m) - nikW . (3) 

The intuitive justification of this approach is that as 
loopy BP converges, the diff'erences between the messages 
before and after an update go to zero. Therefore if a mes- 
sage has a large residual, it means that it's located in a 
part of the graph that hasn't converged yet. Therefore, 
propagating that message first should speed up the pro- 
cess. Elidan et al. create a priority queue, ordered by the 
value of the residual, so at each step the first message in 
the queue is updated and then the queue is reordered using 
the new information. 
A. RBP decoding for LDPC codes 

In LLR BP decoding, all the message spaces are one- 
dimensional (the real line). Therefore, the residuals are 
the absolute values of the difference of the LLRs. 

Let us analyze the behavior of RBP decoding for LDPC 
codes in order to simplify the decoding algorithm. Initially, 
all the messages niy^c are set to the value of their corre- 
spondent channel message Cy. No operations are needed 
in this initialization. This implies that the residuals of 
all the variable-to-check messages rlniy^c) are equal to 
0. Then, without loss of generality, we assume that the 
message uic^^vj has the residual r* , which is the biggest 
of the graph. After rric-^vj is propagated, only residuals 
r{my.^ca) change, with Ca G Af{vj)\ci. 

The new residuals r{my.^ca) are equal to r*, because r* 
was the change in the message ma^vj and Eq. 1 shows 
that the message update operations of a variable node are 
only sums. Therefore, the messages m^^._>c<, have now the 
biggest residuals in the graph. 

Assuming that propagating the messages niy .^ca won't 
generate any new residuals bigger than r*, RBP can be 
simplified. Every time a message ruc^y is propagated, the 
outgoing messages from the variable node v will be updated 
and propagated. This facilitates the scheduling since we 
need only to maintain a queue Q of messages TOc^«, or- 
dered by the value of their residuals, in order to find out 
the next message to be propagated. RBP LDPC decoding 
is formally described in Algorithm 1, the stopping rule will 
be discussed in Section VI. 

There is an intuitive way to see how RBP decoding works 
for LDPC codes. Let us assume that at a certain moment 
in the decoding, there is a check node Ci with residuals 
r{mci^vb) = for every € N {ci). Now let us assume 
that there is a change in the value of the message uiy-^a- 
The biggest change in a check-to-variable message out of 
Ci (therefore the largest residual) will happen in the edge 
that corresponds to the incoming variable-to-check message 
with the lowest reliability (excluding the message m^^._>ci)- 



Algorithm 1 RBP decoding for LDPC codes 



1: 


Initialize all rric^y = 


2: 


Initialize all to„^_>c^ = Cj 


3: 


Compute all r(mc^^) and generate Q 


4: 


Let nic-^v be the first message in Q 


5: 


Generate and propagate ma^vj 


6: 


Set r{mci^vj) = and re-order Q 


7: 


for every Ca G ■N' {vj) \ci do 


8: 


Generate and propagate m^.^ca 


9: 


for every Vb & Af {ca)\vj do 


10: 


Compute r(mc„^^t) and re-order Q 


11; 


end for 


12: 


end for 


13: 


if Stopping rule is not satisfied then 


14: 


Position=4; 


15: 


end if 



Let us denote by Vk the variable node that is the destina- 
tion of the message that has the largest residual r{mci^vk)- 
Then, the message m^^^c; has the smallest reliability out 
of all messages my^^^a , with Vb €Af (cj) \vj. 

This implies that, for this particular scenario, once 
there's a change in a variable-to-check message, RBP will 
propagate first the message to the variable node with the 
lowest rc^Iiability. This makes sense intuitively. In some 
sense the lowest reliability variable node needs to receive 
new information more than the higher reliability ones. 

The negative effects of the greediness of RBP are appar- 
ent in the case of unsatisfied check nodes. RBP will sched- 
ule to propagate first the message that will "correct" the 
variable node with the lowest reliability. This is the most 
likely variable node to be in error. However, if that variable 
node was already correct, changing its sign will likely gener- 
ate new errors, making the BP convergence more difficult. 
This analysis helps us see why RBP corrects the most likely 
errors faster but has trouble correcting "difficult" errors as 
will be seen in Section V. We define "difficult" errors as 
the errors that need a large number of message updates to 
be corrected. 

B. Node-wise RBP decoding for LDPC codes 

In order to obtain a better performance for all types of 
errors, perhaps a less greedy scheduling strategy must be 
used. As noted earlier, some of the greediness of RBP came 
from the fact that it tends to propagate first the message to 
the less reliable variable nodes. We propose to simultane- 
ously update and propagate all the check-to- variable mes- 
sages that correspond to the same check node c^, instead 
of only propagating the message with the largest residual 
r{mci^vj)- It can be seen, using the analysis presented 
earlier, that this algorithm is less likely to generate new er- 
rors. We call this less greedy strategy node- wise RBP and 
it's performance can be seen in Section V. Node-wise RBP 
is similar to LBP; it is a sequence of check-node updates. 
However, unlike LBP, which follows a pre-determined or- 
der, the check node to be updated next is chosen dynam- 
ically according to the residuals of the check-to-variable 



messages. Node- wise RBP is formally described in Algo- 
rithm 2. 



Algorithm 2 Node-wise RBP decoding for LDPC codes 

1: Initialize all TOc^t, = 

2: Initialize all m^.^a = Cj 

3: Compute all r{mc^v) and generate Q 

4: Let rric-^vj be the first message in Q 

5: for every Wfc e A/'(ci) do 

6: Generate and propagate ma^v^ 

7: Set r{mci^Vk) = and re-order Q 

8: for every Ca & M (wfe) \ci do 

9: Generate and propagate niy^^ca 

10: for every Vb € M (ca) \vk do 

11: Compute r(mc„^„^) and re-order Q 

12: end for 

13: end for 

14: end for 

15: if Stopping rule is not satisfied then 

16: Position=4; 

17: end if 



Node- wise RBP converges both faster than SSS (in terms 
of number of messages updated) and better than SSS (in 
terms of PER of the code for a large number of iterations). 
We can explain intuitively and demonstrate experimentally 
that the lower error rates are achieved because informed 
scheduling allows the LDPC decoder to overcome many 
"trapping sets". Trapping sets, or near-codewords, as de- 
fined in [15], are small variable-node sets such that the 
induced sub-graph has a small number of odd degree neigh- 
bors. In [15], Richardson also mentions that the most trou- 
blesome trapping set errors are those where the odd degree 
neighbors have degree 1 (in the induced sub-graph), and 
the even-degree neighbors have degree 2 (in the induced 
sub-graph) . 

It is likely that node-wise RBP solves the variable nodes 
in error by sequentially updating the degree- 1 check- nodes 
connected to them. When a variable node in a trapping set 
is corrected, the induced sub-graph of the variable-nodes- 
in-crror will change as follows. At least one check node 
that was degree- 2, becomes degree- 1 after the variable node 
correction. This check node is likely to be picked as the 
next check-node to be updated by node-wise RBP because 
its messages will have large residuals. This update will 
probably correct another variable node in the trapping set. 

We corroborated this analysis by studying the behavior 
of the decoders for the noise realizations that the SSS de- 
coder could not solve for 200 iterations and that node-wise 
RBP solved in a very small number of iterations (less than 
10). We found that a large majority of the SSS errors in 
these cases are caused by trapping sets that node- wise RBP 
solved in the manner mentioned before. 

IV. Complexity and Implementation 

Given the surge in popularity of LDPC codes for practi- 
cal implementations, it is interesting to address some issues 



about the complexity of RBP and node-wise RBP when 
compared to SSS and flooding. 

A. Complexity per iteration 

For an IDS we consider one iteration to have occurred 
after the number of updates equals the number of updates 
completed in an SSS or flooding iteration. For RBP, an 
iteration will be counted after the number of check-to- 
variable message updates equals the number of edges in 
the LDPC graph. For node-wise RBP an iteration will be 
counted after a number of check-node updates equals the 
number of check-nodes of the code. 

In [9], the authors prove that if the appropriate up- 
date equations are chosen, the total number of operations 
per iteration of all the sequential schedules is equal to the 
number of operations per iterations of the flooding sched- 
ule, making their complexity per iteration equal. Given 
that both RBP and node- wise RBP arc forms of sequential 
updates, a sequence of message updates in the first case 
and a sequence of check-node updates in the second, then 
they also use, on average, the same; number of message- 
generating operations per iteration (using our definition of 
iteration for informed schedules). 

In order to maintain the same complexity, we use the 
typical stopping rules in the decoding. Stop if at the end 
of an iteration the decoded bits satisfy all the parity-check 
equations, or a maximum number of iterations is reached. 

On top of the message-generation complexity, informed 
schedules incur two extra processes: residual computation 
and ordering of the residuals. As defined in Section III, the 
residual computation requires the value of the message that 
would be propagated. This requires additional complexity 
since there will be numerous message computations that 
will only be used to calculate residuals. We propose to use 
the min-BP check-node update approximation explained in 
[13] to compute the approximate-residual as follows. 



(mfc) 



f k (m) - ml 



(a) 



(4) 



where the superscript (a) stands for approximate and in- 
dicates the min-BP approximation. The min-BP check- 
node update consists of finding the two variable-to-check 
messages with the smallest reliability. Then, the small- 
est reliability is assigned as the check-to- variable message 
reliability for all the edges except the one where the small- 
est reliability came from. The second smallest reliability 
is assigned to that remaining edge. The proper sign is 
computed for all the check- to- variable messages. Thus, re- 
placing all the residual functions for approximate-residual 
functions in Algorithms 1 and 2, defines Approximate RBP 
(ARBP) decoding and node-wise ARBP decoding. These 
significantly simpler algorithms perform as well as the ones 
presented in Section III, as will be seen in Section V. Note 
that we only propose to use min-BP for the residual com- 
putation. For the actual propagation of messages we use 
the full update equations (1) and (2). 

Even for node-wise ARBP, the practical trade-off be- 
tween the increase in the per-iteration complexity and the 
decrease in the number of iterations (and improved FER) 



is difficult to address in general as it depends on specific 
implementation choices. Our current research is addressing 
this trade-off in detail. 

B. Parallel Decoding 

The possibility of having several processors computing 

messages at the same time during the LDPC decoding has 
become an intense area of research and an important rea- 
son why LDPC codes are so successful. Furthermore, codes 
with a specific structure have been shown to allow SSS de- 
coding while maintaining the same parallelism degree ob- 
tained for flooding decoding [4]. In principle, the idea of 
having an ordered sequence of updates, that uses the most 
recent information as much as possible, isn't compatible 
with the idea of simultaneously computing messages. How- 
ever, since the ordering of the queue Q based on the new 
results occurs after the update, it is possible that the sev- 
eral parallel processors can work on different parts of the 
graph while still using the most recent information. 

We define the parallel node- wise ARBP scheduling strat- 
egy as the node- wise ARBP strategy where instead of up- 
dating only the check-node with the largest approximate 
residual, p check-nodes arc updated at the same time. The 
p nodes that have the largest approximate residuals are 
updated simultaneously. These p check nodes are not de- 
signed to work in parallel, unlike the p check-nodes of a 
p X p sub-matrix as defined in [4]. 

However, parallel processing may be implemented ex- 
tending the hardware solutions presented in [16]. For in- 
stance, if one or more check-nodes have in common one 
or more variable nodes, they will all use the same previ- 
ous information and compute the incremental variations 
that are afterwards combined in the variable-node update. 
There are hardware issues, such as memory clashes, that 
still need to be carefully addressed when implementing par- 
allel node- wise ARBP. 

Parallel node-wise ARBP has a very small performance 
degradation when compared with node-wise ARBP, as will 
be seen in Section V. We defined and simulated the paral- 
lel version of node- wise ARBP since it's the simplest IDS 
strategy and therefore, the most likely to be implemented. 

V. Simulation Results 

This section presents the AWGN performance of the dif- 
ferent scheduling strategies presented above. All the simu- 
lations are floating point and use the same rate- 1/2 LDPC 
code. The blocklength of the code is 1944 and it has the 
same sub-matrix structure as the one presented in [4] with 
sub-matrix size equal to 54x54. 

The SSS results shown correspond to the sequential 
check- node update introduced in [4]. This scheduling is 
known as Layered Belief Propagation (LBP) and guaran- 
tees a parallelism degree equal to the sub-matrix size of the 
LDPC code (54 in our case). As shown in [9], different SSS 
strategies produce almost identical results so its selection 
doesn't significantly affect the performance of the decoder. 

Fig. 1 shows the performance of the scheduling strategies 
discussed above, flooding, SSS (LBP), RBP, ARBP, node- 
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Fig. 1. FER Performance of flooding, SSS (LBP), RBP and node- 
wise RBP vs. number of iterations for a fixed Ef,/No = 1.75 
dB 



wise RBP, node- wise ARBP, and parallel node- wise ARBP, 

as the number of iterations increases. The figure shows 
that RBP has a significantly better performance than SSS 
(LBP) for a small number of iterations, but a sub-par per- 
formance for a larger number of iterations. Specifically, the 
performance of RBP at 4 iterations is equal to the perfor- 
mance of SSS (LBP) at 13 iterations, but the curves cross 
over at 19 iterations. This suggests that RBP has trouble 
with "difficult" errors as discussed earlier. 

Node-wise RBP, while not as good as RBP for a small 
number of iterations, shows consistently better perfor- 
mance than SSS (LBP) across all iterations. Specifically, 
the performance of node-wise RBP at 18 iterations is equal 
to the performance of SSS (LBP) at 50 iterations. The re- 
sults for flooding are shown for comparison purposes, and 
replicate the theoretical and empirical results of [4]- [9] that 
claim that flooding needs twice the number of iterations as 
SSS. 

Fig. 1 also shows the performance of the approximate 
residual schedules and compares them with the schedules 
that use the exact residuals. It can be seen that both 
ARBP and node- wise ARBP perform almost indistinguish- 
ably from RBP and node-wise RBP respectively. We re- 
iterate that the approximate residual diminishes the com- 
plexity of residual computation signiflcantly, thus making 
ARBP, and node-wise ARBP more attractive than their 
exact counterparts. 

Furthermore, Fig. 1 also shows the performance of par- 
allel node-wise ARBP. The relatively small loss in perfor- 
mance when compared to node-wise ARBP is the price 
for the throughput increase resulting from the parallelism. 
The number p of check-nodes processed in parallel was 
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Fig. 2. FER Performance of flooding, SSS (LBP) and node-wise 
ARBP for 15 and 50 iterations vs. Et/A^o 

set to 54, which is equal to the parallelism guaranteed by 
SSS (LBP) decoding this structured LDPC code with sub- 
matrix size equal to 54x54 [4]. 

The FER of node-wise ARBP vs. SNR and for 15 and 
50 iterations (maximum) is presented in Fig. 2. The FER 
of flooding and SSS (LBP) are also presented as references. 
It can be seen that the SNR gap between node-wise ARBP 
and SSS (LBP) is more pronounced for a small number of 
iterations and/or a large SNR. 

Fig. 3 and Fig. 4 show the performance of different 
scheduling strategies for the blocklength 1944 rate-1/2 and 
rate 5/6 LDPC codes selected for the IEEE 802. lln stan- 
dard [17]. These simulations were run for a high number of 
iterations (200) and show that node-wise ARBP achieves a 
better FER performance that SSS (LBP). Fig. 4 also shows 
that even for high rate codes, node-wise ARBP converges 
both faster and better than SSS (LBP). 

VI. Conclusions 

This paper shows that, while maintaining the same 
message-generation functions, IDS can improve the per- 
formance of BP LDPC decoding. 

RBP and its simplification ARBP are appropriate for 
applications that have a high target error-rate, given that 
RBP achieves these error-rates using significantly fewer it- 
erations than SSS. They are also appropriate for high-speed 
applications that only allow a small number of iterations. 
However, for applications that require lower error rates and 
allow larger delays RBP and ARBP aren't appropriate. 

For such applications node-wise RBP and its simplifica- 
tion node- wise ARBP perform better than SSS for any tar- 
get error-rate and any number of iterations. These node- 
wise strategies achieve a lower error-rates by overcoming 




Fig. 3. FER vs. number of iterations of the 802. lln blocklength-1944 
rate-1/2 code using flooding, SSS (LBP), ARBP and node-wise 
ARBP for a fixed Eb/No = 1.75 dB 

trapping set errors that SSS cannot solve. Furthermore 
a paraUel implementation of node-wise ARBP was shown 
to perform nearly as well as the original node-wise ARBP, 
making this informed scheduling more attractive for prac- 
tical implementations. 

The improvement in performance of these informed 
scheduling strategies were also shown for a high-rate code 
(rate 5/6). However, they come with the cost of an increase 
in complexity per iteration due to the residual computa- 
tion and its ordering. The trade-off provided by node-wise 
ARBP between increasing the per-iteration complexity and 
reducing the number of iterations (while also reducing the 
FER for a large number of iterations) requires further in- 
vestigation in the context of specific implementations. 

The ideas presented in this work may be extended to 
other communication solutions that use iterative BP, such 
as turbo codes, turbo-equalization, iterative demodulation 
and decoding of high-order constellations. The extensions 
of the IDS strategies may also prove beneficial for loopy BP 
solutions to problems outside the communications field. 
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